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ABSTRACT 

The Distance Learning Policy Laboratory of the Southern Regional 
Education Board (SREB) and many states and regional organizations are coming to a 
consensus on the principles and goals that should shape distance learning policies. In 
the case of quality assurance, the SREB believes there are four guiding principles 
that states should follow. States should take advantage of state, regional, and 
national networks for sharing high quality offerings, and reciprocity should be 
maximized. States should demonstrate a commitment to innovation and a desire to move 
the best of these practices into mainstream activities. State quality assurance 
policies should focus on the needs of the student, and state policies should be shaped 
with the understanding that "distance” is not a defining characteristic of learning 
processes; rather, it is one among many factors to take into account in designing 
effective programs. Within these guiding principles, the quality assurance 
subcommittee has prepared seven recommendations for consideration by the Distance 
Learning Policy Leadership Group and the SREB. These involve: (1) review of current 

oversight policies; (2) adoption of "best practices"; (3) development of more 
effective mechanisms to learn student views concerning the quality of distance 
learning offerings; (4) extending and formalizing the "free trade zone" concept in the 
electronic campus; (5) working toward consistent, coordinated, unbiased, and 
nonduplicative reviews; (6) accelerating work to include outcome measures in 
accountability mechanisms; and (7) establishing common definitions and state data 
reporting mechanisms. Two appendixes contain a quality practices survey and a list of 
respondents to this survey. ( SLD) 
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About the Southern Regional Education Board 
Distance Learning Policy Laboratory 

At its June, 1999, meeting, the Southern Regional Education Board approved the 
establishment of the SREB Distance Learning Policy Laboratory. Building upon the work of 
the Educational Technology Cooperative and the Electronic Campus, the Policy Laboratory 
seeks to reduce or eliminate existing or potential policy barriers to distance learning activities 
in three broad areas: access, quality, and cost. The Policy Laboratory’s main objectives are: 

• Assessing educational policy issues that are identified as barriers; 

• Establishing policy baselines of current practices, procedures and strategies; 

• Assisting states and institutions as they develop ways to use technology to improve 
quality, expand access, and reduce costs; 

• Establishing trial or pilot efforts with State Partners to test new distance learning 
approaches or strategies; 

• Promoting state-level policy changes via existing SREB organizational arrangements 
and agreements; 

• Developing and testing agreements among institutions and states; 

• Utilizing the regional platform to serve as a clearinghouse for states and institutions to 
discuss policy issues and concerns; and 

• Measuring the implementation of policy changes in the SREB states and widely 
disseminating the results. 

The SREB Distance Learning Policy Laboratory is supported in part by a grant from the 
United States Department of Education Fund for the Improvement of Postsecondary 
Education’s (FIPSE) Learning Anytime, Anywhere Partnerships (LAAP) program. The 
contents of this report were developed under the grant but do not necessarily represent the 
policy of the Department of Education, and you should not assume endorsement by the 
Federal Government. Additional support has been provided by a grant from the Stranahan 
Foundation of Toledo, Ohio. 
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Executive Summary 



The purpose of the Quality Assurance subcommittee is to examine the challenges to state 
and system quality review processes posed by distance learning and to recommend needed 
changes. The recommendations are directed at state and system leadership and staff, but 
have implications for other quality assurance bodies and institutions. 

Traditional regulatory approaches to quality assurance are severely tested in the distance 
learning environment. Most of our accountability structures, for example, are tied to 
political and geographic boundaries and to traditional standards of excellence, based on the 
presence of physical and intellectual resources onsite. In contrast, distance learning, driven 
by the global reach of the Internet, has not only evolved beyond these geographic boundaries 
but is embracing new standards of quality. State and system boards are struggling to balance 
the goals of student access and innovation with traditional consumer protection functions. 
National providers, both public and private, are finding it difficult to operate in the maze of 
federal, state, and accreditation oversight that currently exists. 

The emergence of voluntary “best practice” standards is encouraging, however, and many 
states in the South have adopted national and regional standards. In our survey of state 
quality assurance initiatives, we also discovered a number of other promising actions, 
including the development of state and institutional strategic planning in distance learning. 

States will continue to be challenged by the changing marketplace, and our report calls 
attention to such issues as evaluation of consortia and non-educational vendors, evaluation of 
courses and modules, and dealing with “blended programs.” 

Principles and Recommendations 

The Distance Learning Policy Laboratory of SREB, as well as many states and regional 
organizations, are coming to a consensus on the principles and goals that should shape our 
distance learning policies. In the case of quality assurance, we believe there are four guiding 
principles that states should follow: 

• States should take advantage of statewide, regional, and national networks for 
sharing high quality offerings, and reciprocity should be maximized. 

• States should demonstrate a commitment to innovation and a desire to move 
the best of these practices into mainstream activities. 

• State quality assurance policies should focus on the needs of the student. 

• State policies should be shaped with the understanding that “distance” is not a 
defining characteristic of learning processes; rather it is one among many 
factors to take into account in designing effective programs. 
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Within these guiding principles, the quality assurance subcommittee submits the following 
recommendations for consideration by the Distance Learning Policy Leadership Group and 
SREB. 

1 . States should review their current oversight policies to see if they meet the test of: a) taking 
advantage of regional and national assets; b) promoting innovation; and c) being student- 
centered. This review should take place in the context of the principles described in 
this report, namely toward the goals of developing a “regional free-market” in the 
South and encouraging innovation. As a regional convening body, SREB can greatly 
aid in this process of review. 

2. States and institutions should consider adopting the “best practices ” highlighted in this 
report, particularly those which use peer-review approaches. Our survey of states turned 
up a number of encouraging practices and guidelines that are likely to improve the 
quality of distance learning opportunities. 

3. States should encourage their institutions to develop more effective evaluative mechanisms 
to learn student views concerning the quality of distance learning offerings. Regional 
organizations like SREB, along with state and system-level virtual universities, should 
encourage states to include easily accessible student evaluations in their information 
systems and web sites. The recommendations made by the Pew Learning and 
Technology program for online course evaluation systems are a good starting point 
for designing these evaluation systems. 

4. SREB should extend and formalize its current “fee trade zone ” concept in the Electronic 
Campus, establishing a system of institutional and program reciprocity where approval 
under agreed-upon standards in one jurisdiction is extended to and accepted by others in 
the South. While we do not expect that SREB alone can sort out the complexity of 
overlapping and conflicting standards described in this report, we do believe its 
leadership can establish a precedent that may inspire all parties toward better 
cooperation and more open access to all legitimate providers. States and regional 
accrediting bodies have a special obligation to coordinate their licensing and review 
activities and to establish ongoing structures for dialogue regarding acceptable 
standards in the field of distance learning. 

3. State agencies with responsibilities for oversight of distance learning activities have an 
obligation to work toward consistent, coordinated, unbiased, and non-duplicative reviews. 
Again, we do not expect states to be able to sort out all of the issues raised by entities 
outside of their control (such as the Federal government), but we do believe that 
much progress can be made in quality assurance through communication and the 
development of a greater consensus within the standards-setting community 
operating in a state. 
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6. States and institutions should accelerate their work to include outcome measures in their 
accountability mechanisms for both traditional and distance learning programs. We are 
encouraged by the development of competency-based measures and exams in higher 
education as well as the emphasis placed upon outcome measures by accrediting 
bodies. Many of these efforts are aimed at developing better institutional outcome 
measures; but as was clearly apparent in recent national efforts to “grade” the states on 
student learning, there is a pressing need for more and better indicators. 

7. SREB, through the Electronic Campus and Data Exchange initiatives, should work to 
establish common definitions and state data reporting mechanisms for technology-mediated 
learning activities in the region. A continuing challenge for states is the lack of 
common data collection techniques and even common definitions of the range of 
technology-mediated activity. SREB can build upon the work already underway to 
establish workable procedures to aid in assessment activities. 
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The purpose of the Quality Assurance subcommittee is to examine the challenges to state 
and system quality review processes posed by distance learning and to recommend needed 
changes. Given the close interdependence that exists among federal, state, and voluntary 
accreditation activities, the recommendations found here have some applicability beyond the 
state level, including applicability to private college consortia and individual institutional 
review processes. More fundamental structural change will also depend upon action and 
leadership at the federal level. 

Changing student characteristics, changing societal and higher education environments, and 
increasingly advanced and affordable technologies continue to support the growth of distance 
learning. The flexibility offered to students, as well as the need to efficiently and affordably 
provide a broad range of educational opportunities to growing higher education populations, 
provide strong arguments for its positive future. 

But many tools — including distance learning technologies and practices — can be used for 
good or ill. The growth of distance learning also creates the potential for opportunistic 
providers to engage in poor educational practice and diploma mill-style fraud. The recent 
growth in the number of unaccredited, for-profit providers engaging in questionable 
marketing and content delivery practices raises very serious concerns among higher- 
education leaders. Abuses are not new in higher education, but the ability to use technology 
to efficiently and cheaply reach a broader market is enticing, and doing so can be quite 
profitable. Enabling innovation in the promising field of distance learning — while guarding 
against abuse — presents a real challenge to those charged with responsibilities to ensure 
quality. 

Those same issues challenge potential students, as well. How can they be sure that what they 
are paying for is a “good” course, offered by a “reputable” or “accredited” provider that will 
remain in business over the long term? Those questions are often difficult to answer, even by 
those generally well informed about higher education. While responsibility to police 
diploma-mill activity will fall to state consumer protection entities, it is exceedingly difficult 
to locate and monitor Internet-based activity that can be initiated by any single person with 
access to the World Wide Web and that can occur from any location. The first line of 
defense will, by necessity, be consumer awareness and better information. State higher 
education agencies and institutions will need to play a greater role in educating the general 
public on the determining factors that characterize quality instruction. 

Ensuring that public funds support quality higher education, and providing guidance and 
assistance to those seeking to identify high quality distance learning opportunities, will 
become even more important in the foreseeable future. This report analyzes some of the 
challenges those tasks imply, examines what states are now doing to meet them, identifies 
emerging issues and promising approaches, and makes recommendations about steps that 
should be taken to strengthen quality assurance. 
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The Challenges of Quality Assurance in a Distance Learning Environment 

Quality assurance and accountability for performance are of considerable concern to state 
and federal policymakers and accrediting bodies alike. This concern has taken two forms: 1) 
the desire to improve the effectiveness of all our educational institutions; and 2) the need to 
protect students and the public from fraud and abuse. While both performance 
enhancement and consumer protection are a part of most accountability systems, the 
emphasis in recent years (at least among states and accrediting bodies, if not the Federal 
government) has shifted decidedly in the direction of measuring and increasing the level of 
performance. 

Whether concerns about quality and consumer protection should be any greater in the 
distance learning environment than traditional environments is debatable, but new delivery 
mechanisms face a “burden of proof’ in the minds of many policymakers and traditional 
academics. 

The particular challenges faced in providing effective quality assurance mechanisms in 
distance learning include the following: 

The Boundary Problem 

Most of our current accountability structures are tied to political and geographic boundaries, 
whether they are organized around state, regional, or national lines. The reality of distance 
learning, however, is that it respects no such boundaries. Courses and programs are now 
widely available in a worldwide marketplace of educational providers, often with no more 
effort than the click of a mouse. Governmental agencies and accrediting bodies must not 
only determine the extent to which they can assert their jurisdictional authority, they now 
should (but often don’t) coordinate development of standards as well as the application of 
approval processes and sanctions with a wide array of other players. 

The Standards Problem 

Traditional programs have benefited from widely held assumptions about what constitutes a 
quality educational experience. In general, the presence of certain resources and 
characteristics are presumed to indicate or ensure quality. For example, the employment of 
full-time faculty with appropriate degrees, the presence of an adequate library and physical 
plant, and a system of shared governance are at the heart of traditional accreditation 
standards and program approval policies of states and systems. Yet different standards may 
be needed for distance learning programs which may not have full-time faculty in the 
traditional sense, or even a physical campus. 

To respond to that challenge, a set of standards, less tied to physical assets and more to the 
presence of “good practice,” is evolving to judge acceptable levels of quality distance learning 
as well as conventional instruction. As for outcome measures, both traditional and distance 
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learning programs are equally challenged in providing valuable and relevant information to 
judge learning effectiveness. Higher education’s shortcomings in this regard were readily 
apparent in the national project Measuring Up 2000: The State- by-State Report Card for 
Higher Education, as Measuring Up gave each state an “incomplete” on the “Learning” 
indicator. 

The Regulation Problem 

Regulation, already losing favor in traditional oversight activities, may be especially 
ineffective in guiding distance learning. The Internet — now the medium of choice for the 
majority of distance learning activity — is particularly resistant to governmental regulation, as 
policymakers in other arenas have discovered. Furthermore, in an environment where 
standards of practice are rapidly changing, codification can be an unintentional inhibitor of 
promising innovation. More problematic is the dilemma of enforcement. Given the 
distributed nature of instructional providers and students, state agencies can easily find 
themselves with tough laws that are virtually unenforceable. The postsecondary education 
community needs to be sensitive and watchful of those who seek to take advantage of 
students seeking Internet-based or distance learning. Because distance learning is becoming 
increasingly difficult to monitor, better consumer information may be the best approach to 
combating fraud. SHEEOs and higher-education institutions need to adopt the best 
practices outlined in this report and take an active approach to better educate consumers to 
evaluate quality for themselves. 

The Constitutional Problem 

While not yet an issue for the federal courts, national providers of distance learning, faced 
with multiple standards and regulations from states and accrediting bodies, could potentially 
challenge such oversight on constitutional grounds. The issue, according to Michael 
Goldstein, an attorney and expert on state regulation, hinges on a delicate balance between 
whether education, as it is evolving, is perceived by the courts as a product of “commerce” 
(and therefore protected from unnecessary state control) or is defined as one of the principal 
“reserved powers” granted to the states. As an international commercial product, state 
regulation might well be limited as “restraint of trade” under the U.S. Constitution. For 
now, state regulation of education rests upon the “reserve clause” of the 10' amendment. 

The question then becomes: Even if control of postsecondary education is a “reserved 
power” of the states, when does an institution have enough “physical presence” of the entity 
within the state to permit the state to exercise oversight? In the past this power has limited 
the state regulation, for example, of correspondence programs. States have landed firmly at 
both extremes, some completely avoiding attempts to regulate e-learning, or others enacting 
laws that encompassed electronic instruction. Statutes in some states have now encompassed 
electronic instructional activity even if the institution has absolutely no “physical” or 
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organizational presence in the state — an extension of regulatory authority that may be 
questionable, if challenged. 1 

The Parties and their Roles in Quality Assurance 

Traditionally, the parties to quality assurance have been referred to as the “Triad,” which 
includes the states, the federal government, and the accrediting community. Although each 
of these parties has somewhat different functions and responsibilities, to the institutional 
applicant, their activities often look like a maze of duplicative, and sometimes contradictory, 
standards and processes. Moreover, the “Triad” represents a far more complex array of 
quality assurance and consumer protection oversight than the simple “threesome” conveyed 
by this term. And to a surprising degree the three legs of the “Triad” are interdependent. 
Access to federal aid, for example, depends on having state authorization and federally 
recognized accreditation, while accreditation always requires state authorization as a 
prerequisite. In addition, state authorization statutes frequently delegate to accrediting 
bodies all or many aspects of quality assurance. 

Stresses and legitimate differences of opinion exist within each member of the Triad and 
between them as well. Entities making up the Triad operate in continually shifting 
educational, political, and economic environments that vary state-by-state, region-by-region, 
and, for all geographic divisions, over time. Influences and pressures on the Triad partners 
come from different sources (i.e. faculty seeking to ensure high standards, institutions 
seeking to limit competition, legislators supporting workforce goals, and the business 
aspirations of for-profit providers). The partners may therefore have different underlying 
goals, causing them to react to change in different ways. These realities present a perplexing 
situation for new providers, and for existing providers seeking to do new things. 

The parties involved in the quality assurance process include: 

The U.S. Department of Education . By establishing a wide range of rules and regulations to 
which institutions wishing to participate in federal student aid programs must adhere, the 
Department exercises substantial control over virtually every degree-granting institution in 
the United States. (Only the tiny few who choose to exclude their students from federal aid 
are exempt.) Distance learning providers face additional federal regulations that dictate: 

• the proportion of a degree program’s courses that may be offered via distance learning; 

• the manner through which providers can contract with other organizations and 
companies to market and promote their services; 



1 The statute in Georgia (Georgia Code, Sec. 20-3-250-7) reads “no institution may offer postsecondary education. . .to 
Georgia residents from a location outside this state by... any telecommunications or electronic media technology unless 
issued a current (Georgia) certificate of authority.” In the case of Colorado, an institution is deemed to have a physical 
presence in the state “if it delivers... instruction in Colorado and receives assistance from any other organization within 
the state, such as, but not limited to a cable television company or a television broadcast station that carries instruction 
sponsored by the institution.” (Policies of the Colorado Commission on Higher Education, Sec. 2.04.02) 
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• the manner in which recruiters can be compensated; 

• the amount and manner of instruction that is offered on a weekly and semester basis; 

• the nature of credit load standards and inter-institutional agreements; and 

• the way satisfactory academic standards of progress might be measured. 

Since a lack of access to federal student aid funds is almost always fatal for an institution — 
dramatically limiting an institution’s ability to attract students — these rules (encompassing 
many hundreds of pages in the Code of Federal Regulations) rise to the level of absolute 
obligations. 

The Veterans Affairs Office . By applying 1950’s era laws enacted by the Congress to prevent 
fraud in the use of GI Bill funds, the federal Veterans Affairs (VA) Office exercises approval 
authority over institutions at which veterans choose to enroll and use VA benefits. Today, 
the VA office continues to impose limits on the use of distance learning and maintains 
significant oversight of institutions of higher education. VA standards are established and 
applied separately from those of the U.S. Department of Education. 

State Licensing Agencies . Under the “Reserved Powers” clause of the Tenth Amendment to 
the Constitution, control of education has, for over two centuries, been deemed to lie with 
the states. In turn, state statutes vest the authority to license and/or regulate postsecondary 
institutions across a wide-variety of variously configured state agencies. Some state agencies 
regulating higher education are independent governmental bodies (e.g. New York, 
California); some are within the state departments of education (Florida); some are within 
statewide coordinating boards (Texas); and a few are located within system-wide governing 
boards for public institutions (e.g. North Carolina). Theoretically, an institution must first 
seek and obtain a license to operate in a state before it can seek accreditation or federal 
approval to participate in student aid programs. In practice however, the processes tend to 
be simultaneous. Further, many states provide a wide range of exemptions from licensure 
and re-licensure, most commonly to institutions that are already regionally accredited and 
those with religious purposes. Standards for licensure vary widely, but are increasingly keyed 
to regional accreditation standards. To complicate matters, standards for the licensure of 
out-of-state institutions are often different (and sometimes significantly more strict) than 
those for in-state institutions. These state licensing agencies may also serve as repositories for 
student records in the case of institutional closure, and may establish policies regarding 
security bonds and tuition refunds. 

State Consumer Protection Agencies . Usually resting within Offices of Consumer Affairs or 
under the purview of the state attorney general, these agencies receive consumer complaints 
and enforce the states’ consumer protection legislation. In some cases they include an 
investigative wing to pursue cases of financial aid fraud and other abuses of public funds. In 
at least one state (California), the institutional licensing agency is part of the state consumer 
protection agency. 
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State Coordinating Boards and System-wide Governing Boards . Nearly all states have 
established bodies with the authority to approve new, and in some cases to review existing, 
academic programs. In most cases, this authority extends only to public institutions, but in a 
few cases control extends to private independent institutions as well. These bodies often 
develop rules or policies affecting distance learning. Some states, including states in the 
South, have system governing boards with program approval authority over campuses, as 
well as statewide coordinating boards with similar over-riding approval authority. The 
application of program review policies varies widely, however, even among states with similar 
statutes and regulatory bodies. In practice, state coordinating boards may delegate many 
review responsibilities to the institutions or exempt some institutions and programs from 
review. In recent years, state and system review practices have become less centralized. 

Accrediting Bodies . Like federal oversight by the U.S. Department of Education, voluntary 
accreditation encompasses virtually all of the degree-granting institutions in the United 
States. While a voluntary process, without regional or national accreditation from one of 
these federally approved bodies, institutions are unable to participate in federal aid programs. 
This results in a voluntary process being, de facto, an essential requirement in order to 
operate in the United States. Institutions develop accreditation standards collectively, and 
“peer” representatives from similar institutions conduct reviews. In recent years, the regional 
accrediting community has made concerted efforts to develop relatively common standards 
of good practice and to gain consensus across accrediting bodies. Nevertheless, important 
differences in standards and practice remain, encouraging a certain degree of “forum 
shopping” to find the most amenable accrediting body. 

In addition to regional accrediting bodies, institutions are also subject to review by 
professional accrediting organizations that focus on particular fields or programs (i.e. 
medicine, law, and business). They may, and often do, impose program-specific standards 
on an institution. Finally, “national” accrediting bodies offer accreditation based on various 
approaches to education, or groupings of various disciplines. Some institutions seek 
accreditation by those entities, instead of — or in addition to — regional accreditation. 

Professional Licensure Boards . These are either state entities or organizations acting for the 
state. By limiting individual access to a profession, they control the use of credentials and 
impose standards that can render an academic credential useless. Some professional boards 
may grant limited licensure to graduates of “approved institutions,” and sometimes those 
may only be institutions located within the state. They may impose residency requirements, 
and often exclude “correspondence study” from qualified programs. In terms of the effect on 
distance learning, many of the rules of such boards, dating to a time when the telephone was 
considered a “new” technology, do not differentiate between forms of learning at a distance. 
In concert with professional accrediting bodies, they are a powerful force for imposing 
standards of learning and educational practice. 
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The above suggests that there is a continuing need for coordinating and standard-setting 
among the various players in the quality assurance arena. In the absence of federal action, it 
may — and perhaps should — be left to the states and regional associations to exercise that 
leadership. 

Current Activities of State Coordinating and System Governing Boards 

In order to determine the current status and practice of distance learning oversight by state 
higher education systems and agencies, the Quality Assurance Subcommittee of Distance 
Learning Policy Laboratory surveyed state-level higher education boards in the Spring of 
2001. (See Appendix A for a list of responding entities and Appendix B for a copy of survey 
questions). Responses were received from 35 of the 50 states. Highlights from this survey 
include the following: 

• 83% (29 of 35 respondents) indicated that their agency had “adopted rules or policy 
statements on distance learning expectations.” 

• Most states (57%) have relied upon the standards of good practice promulgated by 
one or more of the regional organizations or accrediting bodies. 

• Thirteen states (40%) have required institutions to develop specific distance learning 
plans. 

• Most states (69%) limit their oversight to stating expectations; less than a quarter 
indicated that they “audit” offerings. 

• When complaints are received about online learning, they are typically referred to the 
offering institution for resolution. 

Strategic Plans . In our survey of states we asked about the existence of both statewide and 
institutional-level plans for distance learning. Fewer than half of the states indicated that 
they had adopted such plans, and only 37% of the states include provisions for quality 
assurance. Even fewer state agencies (26%) require the institutions under their oversight to 
adopt strategic plans, and only 20% require the inclusion of quality assurance mechanisms in 
those plans. When asked what elements a strategic plan should contain, survey respondents 
recommended the following provisions: 

• Mission: An institution’s strategic plan for distance learning should reflect state 
needs as well as system and institutional missions. Plans should address methods 
of inter-institutional collaboration and incorporate reward systems to encourage 
such collaborations where appropriate. 

• Prior Planning: Strategic plans should be based on a thorough needs assessment 
and prior consideration of: 1) technological needs and available resources; 2) 
enrollment projections; 3) market forces and service areas; and 4) financial 
assessments, including costs. 

• A focus on student needs and access, including affordability and support service 
needs 
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• Faculty skill development, including course development and technical support 

• Ready student access to adequate learning resources, including libraries, facilities, 
and equipment 

• Provisions for quality control, including specific benchmarks and indicators to be 
used 

• Minimal standards for: 1) faculty qualifications, 2) faculty-student interaction, 
and 3) courseware/content 

• Specific statements regarding assigned roles and responsibilities for all involved 
parties 

• Specific faculty ownership, intellectual property, and compensation policies 

• Provisions for confidentiality and communications security 

Outcome Measures and Comparative Studies . In the April 2001 survey, only three states 
reported comparing the results of on-campus cohorts to distance learning cohorts. The 
University of Wisconsin System Administration notes that “students in the distance learning 
perform at least as well as on-campus cohorts, if not better.” The University of North 
Carolina Office of the President also reports that off-campus students do at least as well in 
their courses as on-campus students do. But even fewer states compare the performance of 
on-campus students to distance learning students on outcome measures such as state 
licensure exams. Only Texas reported using an outcome measure to evaluate distance 
learning programs — in this case, comparing distance learning students’ performance on 
nursing licensing exams with those of on-campus cohorts. 

Special Distance Learning Approval Processes . Some state agencies require prior approval for 
all distance-learning offerings. Fifty-seven percent report that state agency staff must first 
review and approve proposed offerings before they can be made available to students. Others 
require reviews only in special circumstances. West Virginia, for example, requires staff 
review only for courses submitted to the Electronic Campus. In New York, an institution can 
choose either an individual review of proposed programs or an “Institutional Capability 
Review” (seeweb.nysed.gov/ocue/distance). Other states, such as Wisconsin, Louisiana, and 
Georgia, require a stringent prior review only for whole degree programs, and quality 
assurance for particular courses occurs at the local campus level. 

These practices leave many states, however, with policies that fundamentally are “delivery 
medium neutral.” Other states that adhere strictly to a “geographic service area criteria” in 
traditional programs (e.g. in developing off-campus centers) have waived or ignored such 
requirements in approving distance learning programs. In most cases, new distance learning 
offerings of existing programs are not subject to state review (or in some cases go through a 
shorter and more streamlined process). 

Peer Review . Survey respondents rated peer review as the most effective method of ensuring 
quality. In some states, peer advisory committees have been established to review proposed 
offerings, generally with a principal focus on quality issues. A peer review process is required 
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in a third of the reporting states. Several organizations, including the Kentucky Community 
and Technical College System (KCTCS), have found implementation of a peer review to be 
highly effective in increasing the knowledge and skill of course developers. Survey 
respondents also noted that this approach is most effective when conducted at the campus 
level for several reasons. For one, faculties are typically more critical judges of the quality of 
offerings and interactive teaching. Peer involvement also lends great credibility to the 
process and its outcomes; it also encourages greater collaboration and consultation among 
faculty members. However, faculty peer review programs can be very expensive and time 
consuming to develop and maintain. 

Audits . Quality audits of distance learning offerings are practiced by about 27% of the state 
agencies and are viewed as effective in ensuring quality. The South Dakota Board of Regents 
reports that it has implemented a pilot project of randomly auditing 10% of the distance 
learning course offerings. In some cases, faculty peers from other institutions audit a course 
and critique it to improve quality. 

Peer audits through accrediting agencies may occur when an institution’s distance learning 
offerings constitute a “substantive change” for the institution; in addition, distance learning 
offered by regionally accredited institutions is evaluated within the context of each 
institution’s regular accreditation (generally, ten-year) review. 

Best Practice Forums . Although there was great interest in developing them, few state 
systems or boards currently arrange forums for the sharing of best practices in distance 
learning to supplement the opportunities presented by national or regional professional 
organizations. Oregon has conducted various forums to showcase best practices and 
highlight what several faculty were doing from several disciplines in redesigning courses for 
the Web. Alabama has arranged two such forums, which have been effective in establishing a 
network of persons interested in the topic. Several states (e.g. North Carolina, Ohio, and 
Kentucky) hold annual conferences to bring together faculty involved in distance learning 
and to showcase their efforts. 

Voluntary Standard Setting 

One of the most positive developments in distance learning quality assurance has been the 
emergence of the voluntary standards movement. Initiated by the Western Cooperative for 
Educational Telecommunications (WCET), standards of “good practice” have become a 
staple of regional, state, and institutional quality assurance. (See 

http://www.wiche.edu/telecom/projects/balancing/principles.htm). Numerous states and 
accrediting bodies have adopted WCET or similar principles. In 1997, the Electronic 
Campus of SREB adopted its own version of Principles of Good Practice to be used by states 
as a guideline for listing courses on the Electronic Campus web site. Currently, state partners 
provide a written assurance that offerings listed from their state are following these 
guidelines. By forming an agreement among the participating states, SREB has established a 
“free trade zone” for participating colleges and universities in the 16 states, eliminating the 
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need for institutions delivering these “certified” courses and programs in the regional 
marketplace to seek licensure or approval. (See www.electroniccampus.org) 

Among the other national and regional organizations that have promulgated principles of 
good practice are the following: 

The American Distance Education Consortium (ADEC) 
http://www.adec.edu/admin/papers/distance-learning_principles.html 

The American Federation of Teachers (AFT) 
http://www.aft.org/higher_ed/downloadable/distance.pdf 

The Council of Regional Accrediting Commissions (C-RAC) 
http://www.wiche.edu/telcecom/guidelines.htm 

The Institute for Fligher Education Policy (IFIEP) 
http://www.ihep.com/quality.pdf 

In a review of these standards, the Pew Learning and Technology Program 
(http://www.center.rpi.edu/pewsym/mono3.html) noted the high level of convergence 
among the various standard setting exercises. In 2000, a roundtable of experts-called these 
standards “adequate” rather than “best” practices: “Several of the symposium participants 
pointed out that we in higher education are willing to ‘settle’ way too soon, accepting a level 
of performance that is erratic. There is no concept of ‘world class’. . .of meeting or exceeding 
customer expectations ” (p. 9) 

The Pew Roundtable also noted that all of the standard setting exercises had avoided much 
specificity on inputs (such as class size), or measures of outcomes, suggesting further work 
within the standards setting community. 

Actions by Accrediting Bodies 

Led by the Council on Higher Education Accreditation (CHEA) and a new cooperative 
spirit among the regional bodies, the accrediting community has taken a number of actions 
in recent years to address the challenges posed by distance learning. Each of the regional 
associations, for example, has adopted a “Statement of Commitment” and “Best Practices” 
document that encourages cooperation and institutional excellence in distance learning. In 
the area of outcomes, the regional bodies have steadily moved toward more fully 
incorporating this concept in their standards. 

Equally encouraging is the effort of these bodies to more closely coordinate their institutional 
reviews with those of the states. The Middle States Commission and the New York State 
Education department, for example, are near agreement in this area — a step that will 
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significantly reduce the time and effort required of institutions in that state. (See 
http://www.msache.org/special.html) 

Emerging Issues 

In our review of quality assurance practices, a number of issues emerged as likely problems 
for states and accrediting bodies in the near future. 

Evaluating Consortia and N on-educational Providers of Services 

Not only are institutions developing curricula jointly (e.g. Georgia’s “eCore™” program), 
they are outsourcing many instructional or administrative functions to a variety of vendors, 
including those who are assuming important academic functions such as online tutoring and 
content development. This adds to the complexity of state, system, or accrediting body 
reviews and makes it difficult to determine an accountable “controlling authority.” Should 
quality assurance bodies extend their reviews to these private providers, such as tutorial 
services, call centers, Web-course developers? Or should they continue to limit their reviews 
to the degree-granting entity and assume that this body is exercising appropriate quality 
control over sub-contractors? 

Accrediting bodies have taken a pragmatic “results-oriented” approach to this dilemma. 
Rather than accrediting the vendors, they hold the degree-granting institution accountable 
for the vendors’ actions. 

Evaluating Courses and Modules 

One of the effects of technology has been to disaggregate instructional content and the 
learning experience of students into smaller and smaller units. Educational experiences and 
degree programs are more likely today to be a series of courses, even sub-course learning 
modules, rather than a holistic “college experience.” While traditionalists lament this 
development, the trends are clear. The “product” (content or knowledge) purchased by 
“consumers” (students) may be in much smaller units in the future. This suggests that our 
accountability systems will need adjustment as well. The quality of student experiences can 
vary widely, even within the same institutional and program framework. If we are to gain 
customer feedback for quality assurance to add to our expert or peer reviews, it will perhaps 
have to be done at least at the course level, and perhaps at the smaller “unit” level as well. 

An approach to that problem, as noted in the Distance Learning Policy Laboratory Faculty 
Subcommittee report, is the MERLOT initiative, a consortium of systems and institutions 
established to develop a process for conducting peer reviews of online instructional modules, 
course materials, or “learning objects.” MERLOT (www.merlot.org) now includes hundreds 
of ratings of online instructional materials in more than a dozen disciplines. 
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This effort to provide better information and access to digital materials is also promoted by 
two “standards” efforts: 1) the “IMS” project (Instructional Management Systems project; 
see www.imsproject.org), which includes institutions and publishers; and 2) SCORM 
(Sharable Content Object Reference Model; see http://www.adlnet.org), which is a 
Department of Defense led initiative. SCORM also includes a digital library of learning 
modules managed by the Advanced Distributed Learning (ADL) initiative based at the 
University of Wisconsin. 

Dealing with “Blended” Programs 

Many states have assumed that there is a clear distinction between distance learning and 
traditional face-to-face classroom instruction. This would suggest, then, the need for distinct 
criteria and separate review processes. But a “blended” or “mixed” model of delivery is 
emerging as the preference of both students and faculty. In the “partial Internet” course, for 
example, there may be a significant asynchronous online component combined with face-to- 
face meetings. This suggests that separate processes of approval and review may not be 
necessary, especially as distance learning matures (a view that many accrediting bodies are 
taking). Many institutions are now developing proposals for new degree programs with the 
assumption that the program will be delivered in multiple formats — some sections and 
courses may be face to face, others may be completely asynchronous, and still others may use 
the “blended” model. 

Consumer Information/Feedback Systems 

In its evaluation of current consumer information and feedback programs, the Pew 
Roundtable on Quality Assurance evaluated existing search engines for finding courses on 
the Web. The members noted several shortcomings, including the following: 

• Listings by institution and course number instead of sorts by topic or program 

• No differentiation regarding enrollment requirements (i.e. whether the student 
needs to matriculate in the institution or degree program in order to enroll) 

• No information on face-to-face and prerequisite requirements 

• Poor contact information 

• Insufficient information on course requirements 

• No reporting of outcome measures (e.g. drop, failure, or withdrawal rates) 

In developing information systems for students, institutions and faculty should provide a 
realistic assessment of what it takes for successful completion of a distance learning course 
(e.g. greater self-discipline and higher levels of information “literacy” skills). Such a realistic 
preview is likely to reduce dropouts. 2 (This issue is thoroughly discussed in the Distance 



2 “Realistic Previews” are used in organizations to screen out job applicants who will later be dissatisfied and to better 
prepare new hires for the job experiences ahead of them. Presenting clear, in-depth information on both the positive and 
negative aspects of a job typically increases job tenure, organizational commitment, and job performance. Realistic 
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Learning Policy Laboratory report entitled Anytime , Anyplace Services for the 21 s ' Century 
Student , developed by the Student Services Subcommittee.) 

The Pew Roundtable also recommended that institutions develop more systematic and 
useable consumer rating systems, based not on generalizations but on the presence of 
predetermined elements that experts agree constitute “good practice.” For example, students 
could rate their satisfaction on a Likert scale of 1-5, which would then be translated into a 
“satisfaction index.” (Robert Heterick and Carol Twigg elaborated on this concept in their 
April 1, 2001 newsletter: “The Learning Marketplace.” See 
http://www.center.rpi.edu/Lforum/lm/Apr01.html) 

Among the questions students might be asked are the following: 

1. Was the technology easy to use? 

2. Was it reliable? 

3. Was the course relevant and up-to-date? 

4. How challenging was the course? 

5. Did you receive the help you needed? 

6. Did you receive adequate feedback? 

7. Was there sufficient interaction (with students, with faculty)? 

8. Did you have sufficient access to learning resources? 

9. Were assignments clear? 

10. Did you receive adequate technical assistance? 

Principles and Recommendations 

The Distance Learning Policy Laboratory of SREB, as well as many states and regional 
organizations, are coming to a consensus on the principles and goals that should shape our 
distance learning policies. In the case of quality assurance, we believe there are four guiding 
principles that states should follow: 

• States should take advantage of statewide, regional, and national networks for 
sharing high quality offerings, and reciprocity should be maximized. 

• States should demonstrate a commitment to innovation and a desire to move 
the best of these practices into mainstream activities. 

• State quality assurance policies should focus on the needs of the student. 

• State policies should be shaped with the understanding that “distance” is not a 
defining characteristic of learning processes; rather it is one among many 
factors to take into account in designing effective programs. 



Previews also have been found to lower initial expectation levels, attrition from orientation programs, and job turnover. 
For more information, see Breaugh, J.A., & Starke, M. (2000). Research on employee recruitment: So many studies, so 
many remaining questions. Journal of Management, 26 (3), 405-434. 
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Within these guiding principles, the quality assurance subcommittee submits the following 
recommendations for consideration by the Distance Learning Policy Leadership Group and 



SREB. 



1 . States should review their current oversight policies to see if they meet the test of: a) 
taking advantage of regional and national assets; b) promoting innovation; and c) 
being student-centered. This review should take place in the context of the 
principles described in this report, namely toward the goals of developing a 
“regional free-market” in the South and encouraging innovation. As a 
regional convening body, SREB can greatly aid in this process of review. 

2. States and institutions should consider adopting the " best practices ” highlighted in 
this report, particularly those which use peer-review approaches. Our survey of 
states turned up a number of encouraging practices and guidelines that are 
likely to improve the quality of distance learning opportunities. 

3. States should encourage their institutions to develop more effective evaluative 
mechanisms to learn student views concerning the quality of distance learning 
offerings. Regional organizations like SREB, along with state and system-level 
virtual universities, should encourage states to include easily accessible student 
evaluations in their information systems and web sites. The recommendations 
made by the Pew Learning and Technology program for online course 
evaluation systems are a good starting point for designing these evaluation 
systems. 

4. SREB should extend and formalize its current “free trade zone ” concept in the 
Electronic Campus, establishing a system of institutional and program reciprocity 
where approval under agreed-upon standards in one jurisdiction is extended to and 
accepted by others in the South. While we do not expect that SREB alone can 
sort out the complexity of overlapping and conflicting standards described in 
this report, we do believe its leadership can establish a precedent that may 
inspire all parties toward better cooperation and more open access to all 
legitimate providers. States and regional accrediting bodies have a special 
obligation to coordinate their licensing and review activities and to establish 
ongoing structures for dialogue regarding acceptable standards in the field of 
distance learning. 

5. State agencies with responsibilities for oversight of distance learning activities have 
an obligation to work toward consistent, coordinated, unbiased, and non- 
duplicative reviews. Again, we do not expect states to be able to sort out all of 
the issues raised by entities outside of their control (such as the Federal 
government), but we do believe that much progress can be made in quality 
assurance through communication and the development of a greater consensus 
within the standards-setting community operating in a state. 
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6. States and institutions should accelerate their work to include outcome measures in 
their accountability mechanisms for both traditional and distance learning 
programs. We are encouraged by the development of competency-based 
measures and exams in higher education as well as the emphasis placed upon 
outcome measures by accrediting bodies. Many of these efforts are aimed at 
developing better institutional outcome measures; but as was clearly apparent 
in recent national efforts to “grade” the states on student learning, there is a 
pressing need for more and better indicators. 

7. SREB, through the Electronic Campus and Data Exchange initiatives, should 
work to establish common definitions and state data reporting mechanisms for 
technology-mediated learning activities in the region. A continuing challenge for 
states is the lack of common data collection techniques and even common 
definitions of the range of technology-mediated activity. SREB can build 
upon the work already underway to establish workable procedures to aid in 
assessment activities. 

A Concluding Word 

For many years, serious and dedicated people in colleges and universities have worked to 
develop and provide educational opportunities for students unable to attend a residential 
campus full time. The need for those opportunities will undoubtedly continue to grow. 
Institutions will continue to respond, and oversight bodies (i.e. the Triad) will continue to 
devote attention to assuring quality. The above recommendations build on steps already 
taken, and advances in quality assurance are likely to develop by continuing to combine 
deregulation with the judicious application of enforceable and targeted regulation. 
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April 200 1 Quality Assurance Practices Survey Respondents 

Alabama Commission on Higher Education 

Arizona Board of Regents 

Arkansas Department of Higher Education 

California Postsecondary Education Commission 

Connecticut Department of Higher Education 

Delaware Higher Education Commission 

Florida Community Colleges Learning Consortium 

Idaho State Board of Education 

Iowa State Board of Regents 

Kentucky Council on Postsecondary Education 

Louisiana Board of Regents 

Maryland Higher Education Commission 

Minnesota State Colleges and Universities 

Mississippi Institutions of Higher Learning 

Nebraska Coordinating Commission for Postsecondary Education 

New Jersey Commission on Higher Education 

New York State Education Department, Office of Higher Education 

North Dakota University System 

Ohio Learning Network 

Oklahoma State Regents for Higher Education 

Oregon University System 

Rhode Island Office of Higher Education 

South Carolina Commission on Higher Education 

South Dakota Board of Regents 

State Council of Higher Education for Virginia 

Tennessee Higher Education Commission 

Texas Higher Education Coordinating Board 

University and Community College System of Nevada 

University of Hawaii System 

University of North Carolina Office of the President 

University of Wisconsin System Administration 

University System of Georgia 

Utah System of Higher Education 

West Virginia Higher Education Policy Commission 
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April 2001 Quality Assurance Practices Survey Questions 

Q1 . In general, our agency oversees and has policy-making authority for: (check all that 
apply) 

1 . Our agency oversees all higher education in our state 

2. Our agency oversees all public universities in our state 

3. Our agency oversees all community and technical colleges in our state 

4. Our agency has program approval responsibility for independent colleges 

Q2. Has your agency adopted rules or policy statements on distance learning quality 
expectations that apply to all institutions under your oversight or coordination? 

Q3. Those statements are based on: (check all that apply) 

1 . SREB Principles of Good Practice 

2. Regional accrediting organizations 

3. Policy work in our state 

4. Other 

Q4. Are your agency-level rules or policy statements on distance learning quality issues 
available on your web site? 

Q3. Has your agency developed a strategic plan for distance learning for all the institutions 
under your oversight or coordination? 

Q6. Does that plan include provisions for quality assurance? 

Q7. If applicable, when was that plan adopted 

Q8. If applicable— is that plan available on your web site? 

Q9. Has your agency required each of the institutions (or systems of institutions) to develop 
institution-specific or system-specific plans? 

Q10. If yes, are the institution-specific strategic plans required to address quality assurance? 
Q1 1 . If yes, do you require updates of those plans? 

Q12. If yes, how often? 
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Q13. Whether or not you have a plan that contains them, what are the most important 
elements that you would recommend a strategic plan to contain? 

Q14. Our agency attempts to ensure that institutions meet the state’s quality expectations 
for distance learning by (check all that apply): 

1. Making rules and stating expectations 

2. Having agency staff review proposed offerings 

3. Requiring a peer review 

4. Auditing offerings 

5. Other 

Q15. Either formally or informally, has your agency compared the results of on-campus 
cohorts to distance learning cohorts as to their success on state licensure exams (in fields such 
as nursing, for example)? 

Q16. If so, what in general have you found? 

Q17. If your agency is using “outcome measures” to assess the quality of distance learning 
offerings, please provide a description of those measures or identify a web site where they are 
described. What in general have you found? 

Please evaluate the following items as to their value or importance (in your experience) in 
ensuring, at the state or agency level, high-quality distance learning offerings. 

1 = Very Effective 

2 = Effective 

3 = Somewhat effective 

4 = Ineffective 

5 = Very Ineffective 

Q18. Effectiveness of stating expectations 

(Please comment on how you have used or could use this approach to improve quality) 

Q19. Effectiveness of prior certification by the CAO 

(Please comment on how you have used or could use this approach to improve quality) 

Q20. Effectiveness of prior agency review 

(Please comment on how you have used or could use this approach to improve quality) 

Q2 1 . Effectiveness of prior peer review 

(Please comment on how you have used or could use this approach to improve quality) 
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Q22. Effectiveness of quality audits 

(Please comment on how you have used or could use this approach to improve quality) 
Q23. Effectiveness of arranging showcases of best practices 

(Please comment on how you have used or could use this approach to improve quality) 
Q24. Effectiveness of competition/free market forces 

(Please comment on how you have used or could use this approach to improve quality) 
Q25. Effectiveness of Accrediting Agency review 

(Please comment on how you have used or could use this approach to improve quality) 
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